There has been proposed a technology to acquire an image of the surroundings of a vehicle with a camera to show a driver of the vehicle the acquired image for the purpose of making the driver easily check the surroundings of the vehicle and drive safely. For example, if multiple cameras are mounted on the periphery of a vehicle, and respective images acquired by the cameras are synthesized, a driver can visually recognize an image of a wider range of surrounding area of the vehicle.
To acquire a synthetic image that does not give the driver a feeling of strangeness, i.e., a synthetic image in which any joint between images is not misaligned or any images are not skewed as a result of the synthesis of camera images, it is preferable to install the cameras in the predetermined positions and directions. However, as the cameras are mounted on the vehicle, the installation position and angle of the camera may change due to slackness of the mounted part or vibrations caused by running the vehicle, etc. Therefore, it is preferable to check if there is any misalignment of the mounting positions and angles of the cameras and perform a calibration process to calibrate the installation positions and angles of the cameras before shipment of the vehicle, at the safety inspection of the vehicle, and at the arbitrary timing after the vehicle is put into use.
In the calibration process to calibrate the installation positions and angles of the cameras, an image of a marker (a jig) placed in a prescribed position within a shooting range of the camera is taken, and the position of the marker on the taken image, i.e., coordinates of a feature point of the marker on the image is used. As for the marker used in the calibration process, it is preferable that the marker has a pattern of which the feature point can be easily extracted on a camera image; for example, a checkered-patterned marker is conceivable. In the checkered-patterned marker, the center point of the checkered pattern, for example, can be a feature point.
Conventional technologies of extracting a feature point of a checkered-patterned marker include, for example, a technology to detect border lines between different color regions in the checkered pattern on an image and find a point of intersection between the detected lines, thereby extracting a feature point of the marker.    Patent document 1: Japanese Laid-open Patent Publication No. 2010-087743    Patent document 2: Japanese Laid-open Patent Publication No. 2008-131250
Incidentally, coordinates of a feature point of a marker on an image is used as a parameter of the above-described calibration process. Thus, if the accuracy of extraction of a feature point from a marker on a camera image is low, the accuracy of the calibration process to calibrate the installation positions and angles of cameras is also low. Therefore, it is important to accurately extract a feature point of a marker on a camera image.
For example, as for cameras mounted on a vehicle to take an image of the surroundings of the vehicle, it is preferable to cover the surroundings of the vehicle with as few cameras as possible; therefore, for example, a wide angle camera is often used. The shape of an object seen on an image taken with the wide angle camera tends to gradually become unclear, for example, gradually become smaller or distorted, etc. with distance from the center of the image plane.
When the above-described calibration process is performed, a prescribed position at which a marker is placed can be within an area where shooting ranges of two cameras are overlapped; however, when viewed through each of the cameras, the area is located near the edge of the image plane. Thus, because of the characteristics of the wide angle camera, an image of the marker is unclear, for example, an image of the marker is small or distorted. Furthermore, depending on a condition of the lighting for the marker, there is a case where a checkered pattern of the marker on an image does not look like the checkered pattern. For example, in a case of a marker having a black and white checkered pattern, when the light is strong and bright, a white region in the checkered pattern of the marker seen on an image looks like expanded. On the other hand, when the light is weak and dusky, a white region in the checkered pattern of the marker seen on an image looks like contracted. It is difficult to detect border lines between different color regions in the checkered pattern of the marker from such an unclear marker image and extract a feature point of the marker. Therefore, if an image is unclear, the above-described conventional technology has a difficulty to accurately extract a feature point of a marker.
Incidentally, even when an image of a marker is taken with a camera other than a wide angle camera, the marker image may be substantially distorted at a portion of the image near the edge of the image plane, and also is affected by the influence of the lighting, and therefore, the marker image is highly likely to be unclear. Consequently, even when an image of the marker is taken with a camera other than a wide angle camera, there is a case where the above-described conventional technology has a difficulty to accurately extract a feature point of the marker from an unclear marker image.